Multimodal and mobile conversational Health and Fitness Companions

Multimodal conversational spoken dialogues using physical and virtual agents provide a potential interface to motivate and support users in the domain of health and fitness. In this paper we present how such multimodal conversational Companions can be implemented to support their owners in various pervasive and mobile settings. We present concrete system architectures, virtual, physical and mobile multimodal interfaces, and interaction management techniques for such companions. In particular, we present how knowledge representation and separation of low-level interaction modelling from high-level reasoning at the domain level makes it possible to implement distributed, but still coherent, interaction with Companions. The distribution is enabled by using a dialogue plan to communicate information from domain level planner to dialogue management and from there to a separate mobile interface. The model enables each part of the system to handle the same information from its own perspective without containing overlapping ∗Corresponding author Email addresses: [email protected] (Markku Turunen), [email protected] (Jaakko Hakulinen), [email protected] (Olov St̊ahl), [email protected] (Björn Gambäck), [email protected] (Preben Hansen), [email protected] (Mari C. Rodŕıguez Gancedo), [email protected] (Raúl Santos de la Cámara), [email protected] (Cameron Smith), [email protected] (Daniel Charlton), [email protected] (Marc Cavazza) Preprint submitted to Computer Speech and Language March 25, 2010 Manuscript pe er -0 06 92 18 6, v er si on 1 29 A pr 2 01 2